Papers with iterative decoding
Incorporating a Local Translation Mechanism into Non-autoregressive Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to capture local dependencies among output tokens are not efficient, causing errors of repeated translation. |
| Approach: | They propose a local autoregressive translation mechanism that predicts a short sequence of tokens for each target decoding position instead of one token. |
| Outcome: | Empirical results show that the proposed method achieves comparable or better performance with fewer decoding iterations, bringing a 2.5x speedup. |
Learning to Rewrite for Non-Autoregressive Neural Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translations have poor inference speed but weak recognition of erroneous translation pieces. |
| Approach: | They propose an architecture to explicitly learn to rewrite the erroneous translation pieces. |
| Outcome: | The proposed architecture can achieve better performance while significantly reducing decoding time. |
Mask-Predict: Parallel Decoding of Conditional Masked Language Models (D19-1)
Copied to clipboard
| Challenge: | a masked language model is used to train a model to predict subsets of mangled words . a parallel decoding algorithm can be used to generate translations in a constant number of iterations. |
| Approach: | They propose a model and a parallel decoding algorithm which train a machine to predict any subset of target words . they introduce conditional masked language models (CMLMs) which are trained with a mangled language model objective . |
| Outcome: | The proposed model improves state-of-the-art performance levels for non-autoregressive and parallel decoding models by over 4 BLEU on average. |
Exploring Non-Autoregressive Text Style Transfer (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for text style transfer use autoregressive decoding, but they are slow and low parallelizability. |
| Approach: | They propose a base NAR model by directly adapting the common training scheme from its AutoRegressive counterpart. |
| Outcome: | The proposed model sacrifices performance due to lack of conditional dependence between output tokens . knowledge distillation, contrastive learning, and iterative decoding are employed to improve the model . |
INarIG: Iterative Non-autoregressive Instruct Generation Model For Word-Level Auto Completion (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for word-level autocompletion (WLAC) only use human typed sequences as prefixes in decoding module. |
| Approach: | They propose a novel iterative nonautoregressive instruct generation model for WLAC task . it uses human typed sequences and iterating decoding with subwords to fully utilize input information. |
| Outcome: | The proposed model is more competent in dealing with low-frequency words, and achieves state-of-the-art results on the WMT22 and benchmark datasets. |
LADR: Locality-Aware Dynamic Rescue for Efficient Text-to-Image Generation with Diffusion Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for inference are expensive and lack spatial redundancy . Discrete Diffusion Language Models are a promising paradigm for multimodal generation . |
| Approach: | They propose a locality-aware dynamic rescue method that exploits spatial Markov property of images. |
| Outcome: | The proposed method achieves an approximate 4 speedup over baselines on four text-to-image generation benchmarks. |
Masked Text-to-Audio Flow-Matching and Reward Feedback Optimization (2026.findings-acl)
Copied to clipboard
Rongjie Huang, Dongchao Yang, Wenxiang Guo, Huadai Liu, Xize Cheng, Zehan Wang, Zhou Zhao, Xixin Wu, Helen M. Meng
| Challenge: | Experimental results show that Flow-matching generative models can scale training by increasing data, computational resources, and model size. |
| Approach: | They propose a flow-matching transformer with masked generative modeling for scaling text-to-audio inference-time prediction. |
| Outcome: | The proposed model scales inference-time computations by masking generation and re-predicting them through iterative decoding. |